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We investigate the role of the learning rate in a Kuramoto Model of coupled phase oscillators in 
which the coupling coefficients dynamically vary according to a Hebbian learning rule. According 
to the Hebbian theory, a synapse between two neurons is strengthened if they are simultaneously 
co-active. Two stable synchronized clusters in anti-phase emerge when the learning rate is larger 
than a critical value. In such a fast learning scenario, the network eventually constructs itself into 
an all-to-all coupled structure, regardless of initial conditions in connectivity. In contrast, when 
learning is slower than this critical value, only a single synchronized cluster can develop. Extending 
our analysis, we explore whether self-development of neuronal networks can be achieved through an 
interaction between spontaneous neural synchronization and Hebbian learning. We find that self- 
development of such neural systems is impossible if learning is too slow. Finally, we demonstrate 
that similar to the acquisition and consolidation of long-term memory, this network is capable of 
generating and remembering stable patterns. 



I. INTRODUCTION 

Spontaneous mass-synchronization has been observed 
in several biological systems, such as in the synchronous 
flashing of fireflies [1] |2], the chirping of crickets [5], 
and in the pacemaker cells in the cardiovascular system 
[4]. Within the nervous system, synchronous clustering 
has been reported in networks of neurons in the visual 
cortex [5], olfactory bulb p], central pattern generators 
[3 m 13 ED] as well as in those involved in generating cir- 
cadian rhythms Neuronal synchronization has been 
attributed to play a role in movement [12], memory [T3] 
and epilepsy [HI [TS] . It is clear that in all these examples 
the structure of the neural network must play a crucial 
role in its function. The adaptive development of the 
network structure takes place through the modifications 
of synaptic connections, governed by underlying neural 
learning mechanisms. Such synaptic modifications are 
posited to constitute the neural basis of learning and the 
consequent acquisition of long term memory |161 117j . 

In the nervous system, a neuron integrates inputs from 
other neurons and generates outputs in the form of action 
potentials or spikes when its membrane potential exceeds 
an electrophysiological threshold. In particular, tonically 
spiking neurons are observed to "fire" spikes at regular 
intervals with a particular time period. Although the dy- 
namics of single neurons are essential, complex cognitive 
phenomena emerge from the interactions of many neu- 
rons. In a given neuronal network, neurons that make 
synaptic connections influence one another through ei- 
ther excitation or inhibition. 

Collective synchronization in natural systems has been 
previously modeled by representing them as networks of 
coupled phase oscillators [H [13 [201 HD HH US] • These 
studies assumed a pre-imposed static network struc- 
ture and connectivity. In particular, the influential Ku- 
ramoto Model [2(1 relied on global, all-to-all connectiv- 



ity in which each oscillator affected every other oscillator 
equally. 

Recent theoretical efforts have studied how a network 
may develop in accordance with neural learning mecha- 
nisms in relation to the dynamics of synchronous clus- 
ter formation [Hj |M1 HSj [53 d?]. Neurophysiological 
studies have shown that a synapse is strengthened if the 
pre-synaptic neuron repeatedly causes the post-synaptic 
neuron to fire, leading to the Long Term Potentiation 
(LTP) of the synapse [251 [25] • Symmetrically, Long Term 
Depression (LTD) occurs when the post-synaptic neuron 
does not fire when the pre-synaptic neuron does. Exper- 
imental findings suggest further that learning may not 
depend solely on the rate of spikes at a synapse but 
on the relative timing of pre and post-synaptic spikes 
[23 EH [32 [SS]- According to the Hebbian theory [5i] . 
the strength of the synapse between two neurons is en- 
hanced if they are simultaneously coactive. In this work, 
we represent the relative time between spikes in the pre 
and post-synaptic neurons as the relative phase of a pair 
of coupled oscillators, and in this way the phase of an 
oscillator may be used to represent the time between 
two spikes generated by a given tonically spiking neuron. 
The intrinsic frequency, the frequency of an oscillator in- 
dependent of any influence from other oscillators, shall 
represent the natural firing-rate of a neuron in a network 

[13 [31]. 

Phase oscillator models with slow time-varying cou- 
pling have previously been capable of displaying asso- 
ciative memory properties, while revealing parameter 
regimes for which both synchronized and unsynchronized 
clusters are stable j21j i26j . We explore how synchroniza- 
tion and learning mutually affect one another for both 
slow and fast learning rates. Similar recent models have 
assumed homogeneous networks with equal intrinsic fre- 
quencies [2^. We show, however, that an oscillator net- 
work develops stable synaptic couplings that depend on 
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the relative intrinsic frequencies and on the learning rate, 
as well as on the initial network state. The paper is or- 
ganized as follows: in Sec. |TT] we introduce the model 
endowed with dynamic connectivity. In Sec. |III| and |IV| 
we focus on the scenario when the network learns quickly 
and slowly, respectively. We extend our findings to the 
scenario when the network starts out without any con- 
nections and self-develops due to the mutual interaction 
of synchronization and learning in Sec. |V] We summarize 
our findings and provide perspectives in Sec. |VI[ 

II. THE MODEL 

The Kuramoto Model [20] considers a system of limit- 
cycle oscillators moving through the phases of their cy- 
cles based on each oscillator's instrinsic frequency and its 
interaction with other oscillators: 



where € [0, 27r) is the phase of the ith oscillator and uji 
is its intrinsic frequency. The intrinsic frequencies uji can 
be drawn from a probability distribution ^(w), which is 
assumed to be unimodal and symmetrically distributed 
around its mean. K is an N x N matrix of coupling coef- 
ficients, and F is a coupling function with a period of 2tt. 
Following Kuramoto f5D], we assume = sin((/)). In 

order to measure the degree of synchronization, a global 
order parameter, r, is defined as 



re^Ht)=j;^Y.^'^'^'^- (2) 

It represents the centroid of the phases with r{t) cor- 
responding to the coherence in phases and respre- 
senting the mean phase. Another convenient measure of 
synchronization is given by G [0, 1], the square of the 
modulus of the order parameter. 

If we assume constant and identical coupling coeffi- 
cients, then Kij — K for all This is known as the 
globally coupled Kuramoto Model. Assuming such cou- 
pHng, Eq. ([TJ becomes 



transition) when K > Kc where K(. is a critical cou- 
pling. If the the distribution g{ijj) of intrinsic frequencies 
is Gaussian, with a standard deviation a, then 

The system of coupled oscillators reaches an average 
degree of synchronization that is independent of initial 
conditions, whether the oscillators started out completely 
in phase or distributed over the unit circle [38'. 

Instead of assuming a constant, pre-imposed connec- 
tivity and coupling matrix, we wish to investigate how 
this network develops through neural learning mecha- 
nisms and affects synchronous cluster formation, and vice 
versa. The learning mechanisms discussed above can be 
represented by dynamically varying coupling coefficients 
according to the rule 



^ = 6[G(</>,-0,)-if.,]. (5) 

Choosing G((/)) — acos{4>) renders Eq. ^ roughly 
equivalent to the Hebbian learning rule. Note that 0j 
and 4)j are simultaneously coactive if they are in phase 
and hence representative of LTP. When they are in anti- 
phase, that is, 4>i~4'j = '"'j the condition is representative 
of LTD. We define a to represents a Learning Enhance- 
ment factor. It amplifies the amount of learning if two 
neurons are coactive. When the Learning Rate e is small, 
synaptic modification is slow. In this case, synchronized 
clusters are formed which are usually stable with respect 
to external noise [25] , although we discuss below how the 
stability depends on a. Since such stabilization, as in the 
Hopfield model [33], is reflective of long-term associative 
memory formation |40j . such a representation would be 
expected to yield an important perspective on the mech- 
anisms of learning. 

Combining the models of spontaneous synchronization 
and Hebbian learning, our joint dynamical system is rep- 
resented by 



"dT " ^ TV E ^™ ~ ''^^ 



~dt ^^'^ ^^('^J ~ ■ 

The Kuramoto Model then reduces to a mean-field 
model. Any particular oscillator is sensitive only to the 
mean, global properties of the entire system of oscillators, 
making the detailed configuration of coupled oscillators 
irrelevant. It can be shown |37j that the degree of syn- 
chronization becomes nonzero (in a second-order phase 



= e [a cos(0j - - Kij] . (7) 

The K,j in Eq. ^ is a saturating term which prevents 
coupling coefficients from increasing or decreasing with- 
out bound. A hard bound such as restricting \Kij\ < 1 
as in [25] can effectively limit the steady state values of 
the coupling coefficients to one of the two hard bounds. 
Although such restrictions can account for the memoriza- 
tion of binary data, a soft bound such as the saturating 



3 



term we enlist here can allow the network to possess more 
diverse connectivity. It should be noted that the num- 
ber of parameters appearing in Eqs. ^ and ^ could 
be reduced by means of rescaling time and absorbing e. 
However, since both e and a are meaningful from a neu- 
rophysiological perspective, we choose to leave the equa- 
tions in their present form. 

Whereas previous work focussed on slow learning 
, we explore the network's behavior for both fast 
and slow learning scenarios. We observe qualitatively 
different behaviors depending on the values of e and a. 
Particularly, we observe that there is a critical value of 
the learning rate below which a single synchronous clus- 
ter is formed as in the original Kuramoto Model in Eq. 
([3]). Above this critical value, two synchronous clusters 
emerge (see Fig. |4]). We define a new order parameter, 
r2, as 



III. FAST LEARNING 

Situations where memorization of specific details are 
necessary involve fast learning. Hippocampal conjuctive 
coding in particular, is believed to involve such rapid, fo- 
cused learning |41j . In our joint dynamical system, when 
e > ec, the coupling coefficients can follow the "fixed 
point" and so K*j = acos((/)i — Substituting K*^ 

into Eq. ^ then yields, 



a 



N 



dt 



(10) 



Multiplying both sides of Eq. ( 10 1 by 2 and defining 
(f>i = 2(j)i and uJi = 2uji yields. 



1 ^ 

/g^V' (*) ^ jL y^gi20,(t) 



(8) 



The subtraction of the degree of synchronization for a 
single cluster r is necessary since r is large also for the 
single cluster configuration; r2 is designed to pick out the 
dipole moment of the distribution (i.e. two clusters). 

As we will show, the dynamics of the system depends 
on whether the learning rate parameter e is large or small 
compared to some critical tc- For fast learning, e > Ec, 
the coupling coefficients can adjust themselves rapidly 
enough according to Eq. ([t]) that they follow the "fixed 



N 

a \ ^ 



(11) 



This is equivalent to the global Kuramoto Model in Eq. 
([3]), except that the phases are now in double angles. 
We would therefore expect to find a critical value of the 
learning enhancement factor, ac, at which a second-order 
phase transition to synchronization occurs. Under our 
previous assumptions for g{Lu), 



point" Q!COs( 



j) adiabatically as it oscillates before it follows that 



a synchronized state has manifested. For slow learning, 
e < Ec, the coupling coefficients cannot follow the oscilla- 
tion, and thus they can only depart consistently from the 
initial values once a synchronized state has established it- 
self. 

To estimate the magnitude of ec, we have to compare 
the rate at which Eq. can change with the frequency 
of the term cos(0i — It is clear that Eq. (|7| would 
asymptotically approach a static fixed point with a time 



g{Lo)duj = 1 



9 {^) 



g {lj )duj , 



5(w) 



(12) 



(13) 



Accordingly, comparing with Eq. p|, 
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= 2Kc 



(14) 



cos 



,) 



IS 



ujj I , and so on 



constant of ti = 1/e. On the other hand 
expected to oscillate at a frequency of \uji 
average r2 = 7r/2o' is the time it takes for two oscillators 
starting in phase to have moved 7r/2 out-of-phase where 
they do not influence their mutual coupling coefficient 
any longer. Setting these two time scales equal to one 
another yields. 



2cr 



(9) 



In our study, a ~ 0.1, so that Ec ~ 0.064. It should 
be noted that the argument above only holds when 
the starting coupling coefficients of the network satisfy 
Kij{0) > 0.16 (see Eq. Q). Below K^, the sys- 

tem cannot attain global synchronization at all in the 
slow-learning regime. 



In order to verify the value of we performed a series 
of numerical simulations. All simlulations in this study 
employed an Euler timestep of At = 0.1. In Fig. [T] e 
was set to 1.0, so that e > ec — 0.064, and the network 
consisted of 500 oscillators. Intrinsic frequencies oji were 
drawn from a Gaussian distribution with mean /i = 
and standard deviation cr = 0.1. In this case, according 
to Eq. Q, Kc ~ 0.16. We then varied the value of a 
from 1 towards and obtained a bifurcation diagram re- 
lating the average eventual degree of synchronization to 
the learning enhancement factor. We observe a second- 
order phase transition in a for the joint system similar 
to that of the original Kuramoto model with global all- 
to-all coupling. Critically, this phase transition occurs 
at ac = 0.32 = 2Kc, verifying the theoretical prediction. 
This critical value is robust with respect to varying initial 
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FIG. 1: The degree of synchronization r2'^ as a function of 
the learning enhancement factor a when the initial phases of 
oscillators are uniformly distributed over the circle, e > tc was 
set at a large value of 1. A second-order phase transition is 
observed at = 2Kc = 0.32 for Gaussian intrinsic frequency 
distribution of standard deviation a — 0.1. 



conditions of the phase distribution 0i(O) and connectiv- 
ity K,,{0). 

When a > Ofc, e > and all oscillators do not start 
out in phase, two clusters are formed, which remain 180 
degrees apart from each other in mean phase as shown in 
Fig. [2|\. Analyzing the phase-plane we obtain four fixed 
points for the joint dynamics of Eqs. ^ and (|7|. Two 



of them are stable, corresponding to 



— and 



0j — 0i = TT. Thus, stable states for this system occur 
when pairs of oscillators are either synchronized or anti- 
synchronized with each other, leading to the formation 
of the two anti-synchronized clusters. The other steady 
states, corresponding to a relative phase of ^ and are 
unstable. It follows that for the stable steady states. 



K*j — acos( 



,) « ±a 



(15) 



with K*j w a within a synchronized cluster and K*j w 
—a between two anti-synchronized clusters. As seen in 
Fig. [2J3, the final steady-state values of the couphng 
coefficients for the two clusters, observed in the simula- 
tions, are in excellent accordance with the prediction of 
Eq. (151. The final values of the coupling coefficients 



Kij can also be correlated against the initial relative in- 



trinsic frequencies of oscillators Aw^ 



Here 



it is useful to first relate the relative intrinsic frequencies 
of the oscillators to their final relative phases (Fig. |2p). 
Within a cluster, we can calculate the slope of this rela- 
tionship. The scatter-plot relating the final steady-state 
value of the coupling coefficients to the relative intrin- 
sic frequencies of oscillators also depicts the formation of 
two clusters (Fig. [2|l)). Using the slope of the line in Fig. 
|2p together with the cosine fit in Fig. |2]3, we can again 
match the scatter plot very well. 




FIG. 2: Fast Learning (e > Ec) with a = 1 > Oc. (A) Polar 
plot of the distribution of oscillators. Two stable clusters are 
formed. (B) Final Kij as a function of the final relative phases 
|A<^ij|. The thick line represents the data from simulations, 
dashed curve is the theoretical prediction: A"*j = acos{(jii — 
(C) Relative intrinsic frequencies |Ati;ij| as a function 
of final relative phases | A(/)ij|. (D) Final Kij as a function of 
relative intrinsic frequencies | Acuij \ . Black dots represent data 
from simulations, red curve corresponds to the theoretical fit. 



In the fast learning scenario, the strength of the ini- 
tial network coupling has no effect on the eventual net- 
work structure (one or two clusters formed) or degree 
of synchronization. As shown in Fig. [3] regardless of 
whether we start the network without connections, with 
couphng coefficients K{0) < a, K{0) — a, or K{0) > a, 
the degree of synchronization r2 for two clusters always 
attains the same eventual value. In contrast to the orig- 
inal Kuramoto model of Eq. ^ , here the degree of syn- 
chronization does depend on the initial relative phases 
of the oscillators and on the value of a. When a > ac 
and all oscillators start out in phase, that is, 4)i{0) — 
for i = 1,2, ...TV, then only a single synchronized cluster 
is formed (the second cluster being viable but unpopu- 
lated), and a relatively large value of is observed, while 
r| tends toward 0. As discussed above, ii a < Oc then 
no synchronization can manifest. 



IV. SLOW LEARNING 

In neuroscience, the ability of abstracting generalizable 
properties from specific details is believed to involve slow 
learning mediated by the neocortex [41. . In our model of 
coupled phase oscillators, slow learning occurs when e < 
Cc- Qualitatively, since there is very little change in Eq. 
([7]), Kij « Kij{0) = K > Kc on an intermediate time- 
scale. Substitution of this approximate condition into 
Eq. ^ recovers the globally coupled Kuramoto Model 
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FIG. 3: The time evolution of the degree of synchronization 
r2 when q > Qc and e is large (fast learning) for different 
initial coupling strengths K{0). a was fixed at 0.5, e = 1 and 
A"(0) was varied from 0, 0.25, 0.5 and 0.75. The degree of 
synchronization r2 attained the same final value regardless of 
the initial coupling strengths. 

given by Eq. ([s]). In this case, only a single synchronized 
cluster should form, and this result is easily verified by 
simultaions. 

Whether this single cluster remains stable over long 
time scales depends on the value of a. If a is chosen too 
low, an interesting phenomenon occurs whereby a cluster 
initially forms but at long times disintegrates again. The 
eventual disintegration is due to the decrease of the cou- 
pling coefficients at longer times below a value needed to 
sustain synchronization. 

Figure [4] summarizes the transition from a one-cluster 
state to a two-cluster state as e is increased above the crit- 
ical value. The blue trace depicts and the red trace r2^. 
The transition from a one-cluster state at small learning 
rates to a two-cluster state for fast learning is evident. 
A starting value of Kij{0) = 0.75 > Kc was used in the 
simuations shown, but other values of K{0) were tested 
as well. Note that the transition between the one-cluster 
and two-cluster state occurs at the predicted Cc = 0.064 
separating slow and fast learning, thus veryfying the pre- 
diction of Eq. (|9|. 



V. SELF-DEVELOPMENT 

We now consider the intruiging case of ^^^^ (0) — for 
all i, j. This means that we start the joint system out 
with no connections between oscillators in order to ob- 
serve how a connective structure may self-develop in this 
model. In neuroscience terms, we study whether parts 
of the nervous system can develop from the time of con- 
ception through the mutual interaction of spontaneous 
neural synchronization and Hebbian learning in order to 
perform their rich repertoire of functions. 

Let us investigate the role of the learning rate e in 



FIG. 4: The degree of synchronization a function of e when 
K{0) = 0.75. a > ac was set at a large value of 1. Blue and 
red traces represent the average eventual degree of synchro- 
nization for a single cluster and that for two clusters rf, 
respectively. Each data point was computed by simulating a 
network of 250 oscillators for 5000 timesteps and averaging 
over the last 1000 timesteps. 
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FIG. 5: Degree of synchronization as a function of learning 
rate e when K{0) — 0. Red squares represent the numerical 
result for r2 (computed as in Fig. [4]). The black line rep- 
resents the predicted percentage of synchronized pairs. Blue 
dots represent the mean degree of sync for one cluster . 

the self-development of synchronized clusters. For this 
purpose, the learning enhancement factor, a, is set to a 
value well above Uc — found earlier (see Fig. [T| . We 
would like to find the conditions that allow two oscillators 
that are near in phase at some instant of time to become 
entrained to one another. It is clear that in order for this 
to happen, 

kdt>K^,^\\hi (16) 
K w eQ;cos(Awi). 

T denotes the time it would take for the unsynchronized 
oscillator-pair to diverge in phase by 27r and thus meet 
again; i.e. T — 2^/^10. Note that the distribution of 
frequency differences of oscillator-pairs is also normally 
distributed, but with a standard deviation increased by 
a factor of ^/2. 
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This condition implies that the two oscillators can- 
not be any further apart in intrinsic frequency than 
Ao; ~ ea/Kc- Thus, a first estimation of the percent- 
age of oscillator pairs which are able to synchronize is 
given by the following function of e: 



(17) 



This relationship suggests that the degree of synchro- 
nization should depart roughly linearly from the origin 
as e is raised from zero, indicating the absence of a phase 
transition in this case. This prediction is confimred by 
numerical simulations. Figure [sjshows the computed one- 
and two-cluster order parameters, and r\, as a func- 
tion of e. We observe that the one-cluster state does not 
occur for any value of e; it is 'frozen' out for the initial 
condition K(^) = 0. In contrast, the two-cluster state 
gradually turns on as e is increased from zero. 

In order to characterize the coupling coefficients that 
result from a self-assembled network further, let us ex- 
amine the case e = 0.05 (and a = 1, as before). Figure 
[6] depicts a scatter plot of the final coupling coefficients 
between all pairs of oscillators. We observe that the dis- 
tribution falls into two distinct groups. The synchronized 
(and anti-synchronized) oscillator-pairs fall into the top 
and bottom arches. For the unsynchonized oscillators, 
an envelope (see green line in the figure) can be derived 
as follows: 

Since for this sub-population, d4>ijdt w Wj, after sub- 
stition into Eq. ([7|, we obtain the first-order non- 
homogeneous differential equation 



Kij -I- eKij = eacos((/)i — (/)j) = eacosdAwijl^), (18) 
with solutions of 



cos(|Awij|t-H (5), (19) 



where 5 is a phase off-set. Thus, for unsynchronized os- 
cillators, the relation between the coupling coefficients 
attained and the relative intrinsic frequencies remains 
within an envelope (Fig. [6] green trace) that is expressed 
by the amplitude term in Eq. (19 1. For synchronized os- 



cillators, the previous relationship holds (red-fine). 

Whereas previous research demonstrated the ability of 
slow learning co-evolving networks to possess associative 
memory properties and learn binary patterns |24l I26j . 
here we provide a mechanism whereby the network gen- 
erates and learns more diverse patterns even if learning is 
fast. Figure ([t]) illustrates what happens when the learn- 
ing rate e is changed abruptly from a high to a low value. 
This discontinuity happens at i = 1000 in the figure. The 
color in this density plot indicates the phase of the 100 
oscillators relative to the middle one. We see that once 
fast learning establishes a stable pattern, the switch to 




FIG. 6: Final coupling coefficients, Kij, in a self-developed 
network plotted against |ALJij|. Black dots represent simula- 
tion results with A'^ = 250 oscillators, a = l,e = 0.05. Pairs 
of unsynchronized oscillators remain within an envelope while 
other oscillators form two synchronized clusters in anti-phase 
with each other. 



slow learning does not alter this pattern. Stable learning 
of this kind has been put forward as a neurally plausi- 
ble mechanism for the acquisition of long-term memories, 
especially with regard to the consolidation of declarative 
memories. According to the Complementary Learning 
Systems framework [41j . rapid, focused learning enables 
the acquisition of specific memories and their storage in 
the hippocampus, while the neocortex mediates the grad- 
ual extraction of structure and general properties, leading 
to the consolidation of these memories. 
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FIG. 7: The learning rate is abruptly changed from e = 0.1 
to e = 0.01 at a timestep of 1000. The color indicates the 
oscillator phase relative to the middle oscillator (50). Once 
a stable pattern is established with fast learning, it does not 
change when the learning rate is reduced below e^. 



VI. CONCLUSION 

In summary, we have explored the mutual effects of 
spontaneous synchronization and Hebbian learning in a 
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neuronal network, focusing specifically on the role of the 
learning rate. Our work predicts qualitatively different 
behaviors of the network depending on whether learn- 
ing is fast or slow. Specifically, unless the network is in 
a pre-existing state of phase synchrony, when learning 
is fast, it evolves into two anti-synchronized clusters as 
long as a learning enhancement factor, a, is larger than 
a critical value. We found that ac = ^Kc- Further- 
more, when learning is fast and a > ac,the network al- 
ways organizes itself into an all-to-all coupling structure 
with two clusters, regardless of its initial connectivity. 
Such anti-synchronized clustering is observed in neural 
systems known as Central Pattern Generators (CPGs) 
involved in mediating rhythmic motion |7l [3 El [10] . In 
these systems, when a group of neurons burst or fire in 
synchrony together, another group of neurons are inhib- 
ited to quiescence and vice versa. Whereas previous re- 
search has used phase oscillator models to account for 
such reciprocally inhibitory firing synchrony ^9, ID], this 
study suggests how such networks may develop in the 
first place. 



We also predict a critical value in the learning rate, Ec, 
below which learning can be thought of as slow. In this 
regime, for sufficiently strong initial couplings, only one 
synchronized cluster forms. This synchronized cluster 
is stable and is maintained only if a is greater than its 
critical value. Otherwise, the network attains a state 
of synchrony but in the long-term returns to a state of 
disorder. 

Finally, we extended our analysis to the case when a 
network starts out without any connections (or with suf- 
ficiently weak connections). We demonstrated that the 
degree of synchronization varies continuously with the 
learning rate and no phase transition is observed. Here 
when the learning rate is too slow, the network remains 
in an unsynchronized state indefinitely. Thus, our model 
predicts that if learning is too slow, a neuronal network 
cannot self-develop through the mutual interactions of 
neural synchronization and Hebbian learning. In such 
a case, pre-existing couplings, or pre-existing synapses, 
which are sufficiently strong, are necessary for the neu- 
ronal network to self-develop. 
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